Phosphodiesterase 5 (PDE5) hydrolyzes cyclic guanosine monophosphate (cGMP) leading to increased levels of the cAMP response element binding protein (CREB), a transcriptional factor involved with learning and memory processes. We previously reported potent quinoline-based PDE5 inhibitors (PDE5Is) for the treatment of Alzheimer's disease (AD). However, the low aqueous solubility rendered them undesirable drug candidates. Here we report a series of novel PDE5Is with two new scaffolds, 1,2,3,4-tetrahydrobenzo[b][1,6]naphthyridine and 2,3-dihydro-1H-pyrrolo[3,4-b]quinolin-1-one. Among them, compound 6c, 2-acetyl-10-((3-chloro-4-methoxybenzyl)amino)-1,2,3,4-tetrahydrobenzo[b][1,6]naphthyridine-8-carbonitrile, the most potent compound, has an excellent in vitro IC50 (0.056 nM) and improved aqueous solubility as well as good efficacy in a mouse model of AD. Furthermore, we are proposing two plausible binding modes obtained through in silico docking, which provide insights into the structural basis of the activity of the two series of compounds reported herein.